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ABSTRACT 

A monte carlo simulation was conducted to assess the 
effects in an adaptive testing strategy for test batteries of varying 
subtest order, subtest termination criterion, and variable versus 
fixed entry on the psychometric properties of an existent achievement 
test battery. Comparisons were made among conventionally administered 
tests and adaptive tests using adaptive intra-subtest item selection 
with and without inter-subtest branching. The addition of 
inter-subteet^tfranching resulted in levels of mean test battery 
information more similar to those of the full test battery, even with 
mean test battery reductions of 50 percent in number of items 
administered. Subtest order was shown to have no effect on the 
evaluative criteria employed. The results generally supported 
previous studies of this adaptive testing strategy. Suggestions for 
future research are presented. (Author/GK) 
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Factors Influencing the Psychometric Characteristics 
of an Adaptive Testing Strategy for Test Batteries 



Recent studies (e.g., Bejar, Weiss, & Gialluca, 1977; Bejar, Weiss, & 
Kingsbury, 1977) have demonstrated the feasibility of applying unidimcnsional 
adaptive testing strategies to measure classroom achievement. Frequently, how- 
ever, achievement test batteries are composed of items drawn from several dis- 
tinct content or subject matter areas. Under these circumstances, the unidimen- 
sionality assumption may be untenable or inappropriate when applied to the en- 
tire set of items. Application of a unidimensional item response theory (IRT) 
model to multidimensional test data usually results in achievement level esti- 
mates that reflect achievement on onlv a small subset of the test domain (i.e., 
that subset of items having loadings on the first factor). Thus, for example, 
although Be jar, Weiss, and Kingsbury (1977) showed that a single factor emerged 
from analysis of a multicontent achievement test, that factor accounted for only 
23.3% of the variance of the original variables. As a consequence, unidimen- 
sional adaptive testing using that single factor would leave 76.7% of the origi- 
nal variance of the items unaccounted for. Similar results were reported by 
Reckase (1978), who showed first factors of achievement tests accounting for 
1.6% to 81.4% of the variance of the original items. 

Treating multicontent achievement test batteries as if they were unidimen- 
sional also results in loss of diagnostic information about a student by reduc- 
ing data on a student to only one score. Frequently, however, scores on each 
content area are important information to be used by instructors for instruc- 
tional decisions at both the individual and class level. 

Brown and Weiss (1977) designed an adaptive testing strategy for use with 
test batteries that would reduce testing time, yet provide scores on the sepa- 
rate subtests in the battery. Their testing strategy included provision for 
adaptively branching between content area subtests as well as adaptive item se- 
lection within a content area subtest. Brown and Weiss investigated the charac- 
teristics of their combined inter-subtest/intra-subtest adaptive testing strate- 
gy using a real-data simulation of a military achievement test battery. In this 
approach, item response data obtained under conventional testing conditions were 
reanalyzed as if they were administered by the adaptive testing strategy. The 
findings indicated average reductions in test battery length of approximately 
50% while maintaining high levels of psychometric information. Unfortunately, 
Brown and Weiss 1 s (1977) results confounded the relative contributions of intra- 
subtest item selection and inter-subtest branching. 

In an attempt to isolate the separate effects of tbz intra-subtest and in- 
ter-subtest components of the Brown and Weiss testing paradigm, Gialluca and 
Weiss (1979), also using a real-data simulation design, applied the testing 
strategy to a five-subtest biology test battery. Their findings showed test 
length reductions of 20% to 30%, again with minimal loss of psychometric infor- 
mation. They concluded that most of the reduction in test length was due to the 
adaptive intra-subtest item selection procedure and that the addition of inter- 
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subtest branching produced an additional reduction of 1% to 5% in test length. 

Both Brown and Weiss (1977) and Gialluca and Weiss (1979) evaluated the 
performance of the adaptive testing strategy in. similar ways. One of their 
evaluative criteria was correlations of achievement estimates from the adaptive 
tests with those from the original conventional tests. Evaluation of these cor- 
relations is difficult, however, since they are part-whole correlations that are 
artifically inflated due to the items administered in common by the two testing • 
strategies. Thus, although the previous research has demonstrated essentially 
no loss in psychometric information due to the adaptive testing strategy, the 
question of the effect on score validity has not been investigated. Previous 
research also was restricted" to the application of only one method of ordering 
subtests for adaptive administration: subtests in both studies were ordered by 
their multiple correlations with each other. 

Purpose 1 

The present study investigated the separate contributions of intra-subtest 
adaptive item selection and inter-subtest adaptive branching in tetms of (1) 
reduction in the number of items administered, (2) psychometric information 
available in the test scores, and (3) correlations between achievement estimates 
derived from adaptive and conventional test administration with true achievement 
levels. In addition, different methods for ordering the subtests for adaptive 
administration were studied. 



METHOD 
Procedure 

Test Items 

Monte carlo simulation was used so that the validity question, i.e., the 
correlation of observed with true ability estimates, could be adequately inves- 
tigated. Since a simulation study Is valuable to the extent that the underlying 
model accurately reflects the characteristics of actual data, the achievement 
test data used by Brown and Weiss (1977) formed the basis for this study. 

The Brown and Weiss data consisted of the response vectors of 365 Navy fire 
control technicians on a 232-item achievement test battery. The test battery 
was composed of 12 subtests, each covering a different subject matter area. The 
items were parameterized independently for each subtest using Urry's (1976, p. 
99) Ogivia computer pro'*-' am utilizing the three-parameter normal ogive model. 

Table 1 shows the means and standard deviations of the IRT item parameters 
£, l>, and £ from the Brown and Weiss (1977) Ogivia parameterization. (Individ- 
ual item parameter estimates, by subtest, are in Appendix Table A.^ Columns 2, 
3, and 4 of Table 1 show that, from the total available item pool of 232 items, 
item parameter estimates were obtained for 87% (or 201) of these items. Subtest 
12 lost the greatest number of items (28% of the original 25), whereas Subtests 
1, 2, 6, and 8 did not lose any of their items. 

Mean item difficulty (b) ranged from .06 for Subtest 1 to 1.44 for Subtest 



8 



- 3 - 



Table 1 

Means and Standard Deviations of Normal Ogive Item Discrimination (a), 
Difficulty (b), and Pseudo-Guessing (c) Parameters *~ 
for 12 Subtests (from Brown & Weiss, 1977) 



Percent 





Number 


of Items 


of Items 






Parameter 








Avail- 


Parame- 


Parame- 


a 




b 




c 




Subtest 


able 


terized 


terized 


Mean 


SD 


Mean 


SD Mean 


SD 


1 


10 


10 


100 


1.90 


.62 


.06 


1.03 


.52 . 


.11 


2 


10 


10 


100 


2.12 


.86 


.31 


1.29 


.53 


.18 


3 


18 


15 


83 


1.80 


.56 


.54 


1.30 


.55 


.08 


4 


22 


19 


86 


1.60 


.60 


.43 


1.28 


.47 


.08 


5 


18 


17 


94 


1.57 


.65 


.74 


1.32 


.47 


.10 


6 


18 


18 


100 


1.58 


.43 


1.19 


1.45 


.56 


.09 


7 


14 


13 


93 


1.98 


.94 


1.20 


1.26 


.52 


.18 


8 


12 


12 


100 


2.12 


.90 


.84 


1.10 


.43 


.10 


9 


24 


22 


92 


1.49 


.59 


.88 


1.36 


.43 


.10 


. • 10 


29 


23 


79 


1.66 


.57 


1.28 


1.12 


.44 


.14 


X 11 


32 


24 


75 


1.48 


.61 


.91 


1.39 


.43 


.14 


12 


25 


18 


72 


1.73 


.58 


1.44 


1.34 


.52 


.17 


Total 


232 


201 


87 


1.75 


.54 


.82 


1.38 


.49 


.10 



2; and mean item discrimination (a) ranged from 1.48 for Subtest 11 to 2.12 for 
Subtests 2 and 8. Mean estimates of the pseudo-guessing parameter (c) ranged 
from .43 for Subtests 8, 9, and 11 to .56 for Subtest 6. ~ 

Generation of Simulated Examinees 

The first step in a simulation is generation of hypothetical examinees with 
trait levels matching known or assumed characteristics of a real population. 
Using the subtest score intercorrelation matrix from Brown and Weiss (1977, p. 
11), 12 trait levels were generated for each examinee by the following algo- 
rithm. Twelve principal components were extracted from the subtest intercorre- 
lation matrix. Independently distributed standard normal deviates were drawn 
from a random number generator and assigned, one to each component. An examin- 
ee^ trait score was considered to be the sum of the product of the component 
loading for that trait on each of the components, multiplied by the random devi- 
ate nssigned to that component. In matrix notation, this can be written as 

9 = Z W [1] 

where Z is a 1 x 12 matrix of standard normal deviates and W is a 12 x 12 matrix 
of factor loadings. 

A 1,000 x 12 (examinee-by-subtest) matrix of generated true achievement 
levels was obtained for computing the subtest intercorrelation matrix to be used 
for ordering the subtests by multiple correlations. A random sample of 300 ex- 
aminees was retained for the testing simulations. Using this examinee-by-sub- 
test matrix of generated true achievement levels, the multivariate item response 
matrix was generated. 
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Item Response Generation 

Response vectors for simulated examinees were generated based on the item 
parameters for items measuring several latent traits . Given (simulee i's 

true ability on trait k) and the parameters for item £ on trait k, the probabil- 
ity of simulee _i responding correctly was calculated as 

c g k + < l - V T [Da gk < e ik- V ] [2] 

where 

a gk is the discrimination for item g measuring trait k, 

b gk is the difficulty of item g on trait k, 

c gk is the pseudo-guessing level of item g on trait k, 

D is 1.7, a scaling constant for the logistic model, 
and Y[xj is the logistic cumulative distribution function. 

The pi babilities thus computed were compared to a vector of uniformly dis- 
tributed ranuom numbers, r u , in the interval 0 to 1. If 

or if 



r 

u 



> p gk (e ik|' Vk = °-. i*] 

i Conventional Test 

i 

To provide a ba^is for comparison of the adaptive testing strategy, conven- 
tional administration of the achievement battery was also simulated. The sub- 
tests were administered sequentially, with all items within a subtest given in 
order and all examinees taking all the items in the same order. Owen's (1975) 
Bayesian scoring algorithm was used to score each of the subtests, with a mean 
of 0.0 and variance of 1.0 as the initial prior estimate for each subtest. That 
is, no differential item selection or differential subtest order occurred for 
the conventional test. > 



Adaptive Testing Strategy 

As in the Brown and Weiss (1977) and Gialluca and Weiss (1979) studies, an 
adaptive testing strategy utilizing both intra-subtest item selection based upon 
the item information function and a regression-based inter-subtest branching 
rule were used. Three ways of ordering the subtests, for use with the regres- 
sion approach to estimate differential subtest entry, were simulated; (1) or- 
dering on the basis of highest multiple correlation (as in the two previous 
studies), (2) ordering on the basis of number of items in the subtest, and (3) 
random ordering. 

Additionally, an adaptive testing strategy employing only intra-subtest 
item selection was simulated. This was done in order to separate the effects of 
variable termination in the intra-subtest item selection strategy from those of 
the inter-subtest branching strategy. Instead of differential entry into subse- 
quent subtests based on information from previous subtests, each subtest was 
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treated individually, as in the conventional test. As with the conventional 
test, Bayesian scoring was used, with a mean of 0.0 and variance of 1.0 as the 
initial prior e for each of the subtests. Hence, the only difference between 
these tests and the other adaptive tests was that inter-subtest branching (and, 
therefore, a differential Bayesian prior achievement estimate for each siraulee 
on each subtest) was not used. 

Intra-Subtest Adaptive Testing 

Item selection . Adaptive intra-subtest item selection used a maximum in- 
formation item selection method in which the item to be administered to each 
simulee at each stage of the test was that item which provided the most informa- 
tion at that simulee's current achievement level estimate (see Brown & Weiss, 
1977, for a detailed explanation of this method). Hence, for each siraulee, the 
item information value for each unadrainistered itera within a subtest was evalu- 
ated *rora 



where P g (6) is defined as in Equation 2 and P 1 is the first derivative of the 

itera characteristic curve evaluated at 6, the siraulee's current achievement level 
estimate. The itera that had the highest information value w$s selected for ad- 
ministration; once an itera was administered to a siraulee, it was omitted from 
further consideration for that simulee. 

Scoring and termination * As in the conventional test, Bayesian achievement 
level estimates were obtained for each simulee after administration of each 
itera. Termination of testing within a subtest was governed by two criteria: (I) 
testing terminated when all items within a subtest had been administered or (2) 
when no item remaining in a subtest provided a predetermined amount of informa- 
tion at the current estimate of 0. Two minimum values of information were used 
in this study: .01 and .05. 

Inter-Subtest- Adaptive Branching 

Subtest ordering . Three methods of ordering the subtests were simulated. 
In the first method, adapted from Brown and Weiss (1977), subtests were ordered 
on the basis of their highest multiple correlation with each other. Brown and 
Weiss, however, ordered subtests based on linear regression of number-correct 
scores. In this study, the simulee-by-subtest matrix of generated true achieve- 
ment levels (H = 1,000) was in'.ercorrelated and the resulting correlation matrix 
was used as the basis for inter-subtest branching. The highest zero-order cor- 
relation was chosen from this intercorrelation matrix, one of the two subtests 
was arbitrarily chosen to be administered first, and the other was administered 
second. Multiple correlations were then computed using the subtests previously 
chosen first and second as predictors, with each of the remaining subtests, in 
turn, being designated as the criterion. The subtest having the highest corre- 
lation with rhe predictor subtests was chosen to be administered next. By add- 
ing one subtest to the predictor set at each succeeding stage, all subtests were 
thus ordered. 



1(3) 



[p:(§)] 2 



p g (e)[i - yFTi 



[5] 
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Ordering subtests on the basis of this stepwise regression procedure re- 
quires [M(N-l)/2]-N regression equations, whereas given an a priori subtest or- 
dering, only N-l regression equations are required to estimate differential in- 
dividual entry achievement estimates. Although the ordering of subtests by 
stepwise linear regression appears to be a natural adjunct, two other procedures 
were implemented so that the stepwise procedure could be evaluated against al- 
ternatives. 

One alternative, suggested by Brown and Weiss (1977), was to order subtests 
Dased on the number of items. It seems logical to administer the longer sub- 
tests first, since at the early stages of testing a wider range of items would 
enable a more accurate assessment of achievement levels. As the differential 
entry achievement estimates became more accurate, fewer items would be required 
to assess the examinee's achievement levels on s /jsequent tests. 

Subtest order may have an effect on the psychometric characteristics of the 
adaptive testing strategy or on number of items administered. If it does not, 
then it should be possible to obtain the same results by ordering subtests ran- 
domly. To test this hypothesis, subtests were ordered randomly as a third way 
of ordering them for adaptive administration. 

Differential subtest entry . After administration of the first subtest, 
entry achievement estimates for subsequent subtests were differentially deter- 
mined for each simulee. For the first subfest, however, each simulee ! s entry 
achievement estimate was determined by setting its prior achievement level esti- 
mate to be 6 - 0.0 and selecting that item from the initial subtest that provid- 
ed the most information at 9- 0.0. Thus, all simulees began the initial sub- 
test with the same item. 

Entry into the item pool for the next subtest was determined by the bivari- 
ate regression of scores from that subtest on the first subtest and the simu- 
lee 1 s estimated achievement level on the first subtest. This yielded an esti- 
mate of the simulee's achievement level on the second subtest. This achievement 
level estimate then became the Bayesian prior for intra-subtest item selection 
for the second subtest. The squared standard error of estimate from the bivari- 
ate regression equation became the Bayesian prior variance. 

In general, an examinee's final achievement level estimates from all n pre- 
viously administered subtests were used in the appropriate linear multiple~re- 
gression equation for predicting the n + 1st subtest sore. The squared stan- 
dard error of estimate from each regression was^sed as the Bayesian prior vari- 
ance for that subtest. 

Dependent Variables 

V alidity and Test Lengt h 

In a simulation study, true or generated ability or achievement levels es- 
tablish the criterion. Vljus, correlations of achievement level estimates on 
each subtest with their known true values were obtained for all testing condi- 
tions. In addition, the number of items administered under each testing condi- 
tion was examined for each subtest. 

1 o 
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The precision of measurement at different achievement levels is indexed by 
its information value, a quantity that is inversely proportional to the squared 
length of the confidence interval about an achievement level estimate (Hambleton 
& Cook, 1977). Subtest information curves were computed by evaluating the item 
information function at each simulee f s estimated achievement level and summing 
over all items administered in that subtest. Examinees were then grouped into 
equally spaced nonover lapping intervals, and the mean information over all simu- 
lees within an interval was plotted at the midpoint of that interval to obtain 
the subtest information curves. 

Statistics descriptive of information curves . Although graphs are a simple 
and convenient way to present information curves, they are difficult to inter- 
pret directly when many curves are involved. Thus, for comparison purposes, the 
means and coefficients of variation of the information curves were computed.* 

Mean or average information is a statistic that is not disproportionately 
weighted by the distributional characteristics of the examinee group tested. 
The^ greater the mean information, the more precise are the achievement level 
^timates, on the average, over all levels of achievement. The coefficient of 
variation (Snedecor & Cochran, 1967, p. 62), which is equal to the ^tandard de- 
viation o£ the information curve divided by its mean, is of interest because its 
departure from zero indicates that the goal of equiprecise measurement is not 
being met. * 

Relative information . Relative information may be defined- as 

RI A/B " V 9)/ 'V 9) ' • [6J 

where I A (8) is the information value of test A and IgCe) is the information val- 
ue of test B '(Be jar,. 1977). A useful interpretation is in terms of test length 
(Lord, 1980):- when RI A / B >'1.00 at a given achievement level, "test A provides 

an amount of information equal to test B lengthened by a factor of Rl^g. In 

the present study, for example, relative information was computed for each adap- 
tive subtest relative to its conventional counterpart. That is, the numerator 
was the value of information for the adaptive subtest and the denominator was 
the value of information for that same subtest administered conventionally. 



• RESULTS 

Subtest Order 

The intercorrelations of the simulee-by-subtest matrix of generated true 
achievement levels *are shown in the lower triangle of Table 2. The residuals 
from the target correlation matrix (Brown & Weiss, 1977, p. 11) are shown in the 
upper triangle of Table 2. > The largest absolute residual was .08. The average 
absolute deviation from the target matrix was .03, so that, overall, the garget 
matrix appeared to be reproduced faithfully. However, the reconstructed corre- 
lation matrix was sufficiently different to change the rank ordering of the sub- 
tests based on the highest multiple correlations with each other, from the rank 
orderings obtained by Brown and Weiss (1977). The highest bivariate correlation 

JO 
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(♦52) was observed between Subtests 3 and 11, which were designated to be admin- 
istered first and second, respectively. 

Table 2 

Intercorrelations Among True Achievement Levels (Lower Triangle) 
. and Residuals from Generated Correlation Matrix (Upper Triangle) 



Subtest 



Subtest 


1 


2 


■ 3 


4 


5 


6 


7 


8 


9 


10 


11 


12 


1 




-02 


00 


01 


-04 


-05 


-01 


-04 


-01 


-03 


-01 


-02 


2 


29 




-07 


-05 


00 


-04 


-01 


-01 


00 


01 


-05 


06 


3 


40 


30 




-04 


-01 


-05 


00 


-02 


-04 


-02 


-01 


-01 


4 


37 


35 


42 




-02 


-04 


-03 


-05 


-06 


-06 


-02 


-01 


5 


34 


37 


47 


36 




-05 


00 


-03 


00 


00 


-03 


00 


6 


25 


22 


31 


35 


33 




-01 


-06 


-01 


-04 . 


-08 


-01 


7 


29 


37 


41 


33 


46 


34 




-03 


02 


-01 


02 


-01 


8 


21 


28 


27 


23 


32 


24 


33 




-02 


-0'2 


-04 


01 


9 


22 


33 


38 


42 


47 


44 


43 


26 




-05 


-01 


. -04 


10 


16 


36 


25 


27 


28 


29 


32 


25 


35 




-04 


01 


11 


41 


28 


52 


37 


38 


22 


39 


24 


34 


23 




06 


12 


25 


33 


21 


13 


29 


15 


26 


27 


22 


32 


32 





Note * Decimal points omitted* 

Multiple regression-equations were obtained by using Subtests 3 and 11 as 
predictor variables and each of the remaining subtests, in turn, as 'criterion 
variables. This procedure contin; *d until all the subtests had been ordered. 
Table 3 contains the multiple correlations for each subtest predicted from all 
previous subtests and the subsequent ordering of the subtests based on the mul- 
tiple correlations." The raw score regression weights, regression constants, and 
squared standard errors of estimate are showt. in Appendix Table B* Table 4 
shows the ordering sequence of each subtest based on the following three order- 
Table 3 

Multiple Correlations Among Ordered Subtests 



Criterion 
Subtest 










Predictor 


Subtest 










3 


11 


5 


7 


9 


4 


1 


6 


2 


10 


12 


11 


52 






















5 


47 


50 




















7 


41 


46 


54 


















9 


38 


41 


51 


55 
















4 ' 


42 


45 


'48 ' 


49 


53 














1 


40 


46 


48 


48 


49 


57 












6 


31 


32 


38 


41 


49 


51 


51 










2 


30 


33 


41 


45 


46 


48 


49 


49 








10 


25 


27 


32 


36 


41 


42 


42 


42 


47 






12 


21 


32 


37 


38 


38 


38 


40 


40 


44 


47 




8 


27 


29 


36 


39 


40 


40 


40 


41 


42 


43 


44 



Note . Decimal points omitted. 
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ing methoding (1) highest multiple correlation, (2) number of items, and (3) 
random. , Regression weights and related data for the latter two ordering methods 
are in Appendix Tables C and D. 

Table 4 
Order of Administration of 
12 Subtests Resulting from 
Three Ordering Methods 



Ordering Method 



Highest 


Number of 




R 


Items 


Random 


3 


11 


3 


11 


10 


4 


5 


9 


11 


7 


4 


2 


9 


12 


7 


4 


6 


6 


1 


5 


1 


6 


3 


5 


2 


7 


8 


10 


8 


10 


12 


2 


12 


8 


1 


9 



Test Length 

Summary statistics for the number of items administered in each of the 12 
subtests for each subtest order and inter-subtest branching condition, and two 
levels of intra-subtest termination are given in Tables 5 and 6. 

Adaptive intra-subtest item selection . The data in Table 5 summarize the 
reductions in mean test length observed when subtests were administered adap- 
tively but with no inter-subtest branching. That is, each subtest was adminis- 
tered as an independent subtest using only adaptive item selection within each 
subtest. Thus, each examinee began each subtest with a Bayesian prior achieve- 
ment estimate of 0.0 «nd a prior variance of 1.0. The length of the total test 
battery averaged 95.78 items under the termination criterion of .05 and 115.55 
items under the .01 termination criterion. 

The maximum number of items administered for the total battery using only 
adaptive intra-subtest item selection with .05 termination was 154, which was a 
23% reduction in total test length; for the .01 termination condition, the maxi- 
mum number of items administered was 171, which was a 15% reduction in total 
test battery length. The shortest adaptive battery yielded a 78% reduction in 
test length for .05 termination (44 items) and a 72% reduction for .01 termina- 
tion (56 items). The average reduction in test length over the total test bat- 
tery (i.e., the average of the subtest mean reductions weighted by the number of 
items in each subtest) was 52. 3% for the .05 termination and 42.5% for the ,01 
termination. 
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Table 5 

Number of Items Administered in 12 Adaptive and Conventional Subtests 
with Intra-Subtest Item Selection Only at Two Termination Levels 



Termination Criterion of .05 



Termination Criterion of .01 





Conven- 










Percent 




V 
/ 






Percent 


Sub- 


tional 






Range 


Reduc- 






Range 


Reduc- 


test 


Test 


Mean 


SD 


Min 


Max 


tion 


Mean 


SD 


Min 


Max 


tion 


1 


10 


6.07 


1.23 


3 


8 


39.3 


7.29 


.87 


5 


8 


27.1 


2 


10 


6.12 


1.00 


3 


8 


38.8 


7.00 


.80 


4 


8 


30.0 


3 


15 


7.61 


1.42 


3 


a 


49.3 


9.12 


1.50 


4 


12 


39.2 


4 


19 


10.93 


2.20 


6 


15 


42.5 


12.99 


1.89 


6 


15 


31.6 


5 


17 


9.12 


2.30 


5 


12 


46.4 


10.98 


2.60 


6 


14 


35.4 


6 


18 


7.55 


1.99 


3 


10 


58.0 


8.78 


2.03 


4 


15 


51.2 


7 


13 


5.19 


1.60 


2 


12 


60.1 


5.63 


1.42 


2 


12 


56.7 


8 


12 


5.88 


2.05 


3 


10 


51.0 


7.16 


2.14 


3 


11 


40.4 


9 


22 


11.73 


2.68 


6 


17 


46.7 


14.09 


2.90 


8 


19 


35.9 


10 


23 


7.54 


4.14 


3 


21 


67.2 


10.78 


4.58 


3 


21 


53.1 


11 


24 


12.09 


2.58 


5 


17 


49.6 


14.52 


1.82 


8 


20 


39.5 


12 


18 


5.95 


1.99 


2 


13 


66.9 


7.21 


2.14 


3 


16 


60.0 


Total 
























.Battery 


201 


95.78 




44 


154 


52.3 


115.55 




56 


171 


42.5 



♦Computed by the formula 100-[(mean number of items in adaptive test/number 
of items in conventional test) x 100] 

The largest average reduction in subtest length using a termination crite- 
rion of .05 occurred for Subtest 10 and amounted to an average decrease of 67% 
of the items. The smallest average decrease was observed for Subtest 2, which 
showed a 39% reduction. For the .01 termination rule the largest reductioa oc- 
curred for Subtest 12 with an average 60% reduction, while Subtest 1 showed the 
smallest average reduction of 27%. § 

Inter-subtest branching . When the inter-subtest branching strategy was 
employed to implement differential subtest entry in addition Co adaptive intra- 
subtest item selection, test lengths were generally reduced even further. Table 
6 shows the mean test lengths when subtests were ordered by multiple correla- 
tion, by the number or items, and randomly. However, compared to the results 
from intra-subtest it*m selection alone (Table 5), these reductions were slight. 

For example, Table 5 shows that the mean total test battery length when 
using only intra-subtest adaptive item' selection with a termination criterion of 
.05 was 95.78, or a 52.3% reduction. When coupled with inter-subtest adaptive 
branching (Table 6), me«; test battery lengths decreased to 94.10 (53.2% reduc- 
tion) for ordering by multi ~2 correlation, 93.67 (53.4% reduction) for ordering 
by number of items, had i*4.^ (52.8% reduction) for randoftt ordering. For the 
.01 termination criterion, intra-subtest item selection ejlone yielded a mean 
test battery length of 115.55, or a, 42.5% reduction. When inter-subtest branch- 
ing was added, mean test battery lengths of 115.29, 114.A5, and 114.57 were ob- 
served, representing mean test battery reductions of 42.6%, 43.0%, and 43.0% * 
from the full teat battery length. In general (across kll three ordering proce- 
dures), additional reductions in test length due to int/er-subtest branching 
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Table 6 

Number of Items Administered in 12 Adaptive Subtests Ordered 
by Highest Multiple Correlation, by Number of Items, and Randomly, 
Using Differential Subtest Entry 



Ordering 
Method Conven- 

and tional 
Subtest Test 



Termination Criterion of ,05 

Percent 
Range Reduc- 
Mean SD Min Max tion 



Termination Criterion of .01 

Percent 
Range Reduc- 
Min Max tion 



Mean 



SD 



Highest Multiple 
Correlation 



3 


15 


7.55 


1.43 


3 


11 


49. 


7 


9.09 


1.50 


4 


12 


39.4 


11 


24 


11.83 


2.77 


4 


15 


50. 


7 


14.42 


'2.12 


7 


19 


39.9 


5 


17 


9.09 


2.26 


3 


12 


46. 


5 


11.14 


2.44 


5 


14 


34.5 


7 


13 


4.90 


1.38 


1 


11 


62. 


3 


5.74 


1.61 


1 


12 


55.8 


9- 


22 


11.54 


2.92 


3 


17 


47. 


5 


13.96 


3.11 


5 


18 


36.5 


4 


19 


10.75 


2.14 


4 


14 


43. 


4 


12.88 


1.81 


5 


15 


32.2 


1 


10 


6.16 


1.24 


1 


8 


38. 


4 


7.52 


.93 


3 


9 


24.8 


6 


18 


7.13 


1.85 


2 


10 


60. 


4 


8.68 


2.09 


2 


14 


51.8 


2 


10 


5.87 


1.24 


2 


8 


41. 


3 


6.76 


1.00 


3 


8 


32.4 


10 


23 


7.94 


4.09 


2 


21** 


65. 


5 


10.77 


4.84 


2 


21 


53.2 


12 


18 


5.86 


1.69 


2 


12 


67. 


4 


7.18 


2.20 


2 


16 


60.1 


8 


12 


5.48 


2.13 


1 


10 


54. 


3 


7.15 


2.23 


2 


11 


40.4 


Total 


201 


94.10 




28 


149 


53. 


2 


115.29 




41 


169 


42.6 


Number 


of Items 






















11 


24 


11.93 


2.62 


6 


16 


50. 


3 


14.50 


2.01 


8 


20 


39.6 


10 


23 


7. '81 


3.92 


2 


20 


66. 


0 


10.48 


4.69 


2 


21 


54.4 


9 


22 


11.51 


2.90 


5 


16 


47. 


7 


13.99 


3.07 


7 


' 19 


36.4 


4 


19 


10.64 


2.23 


4 


14 


44. 


0 


12.83 


2.04 


5 


15 


32.5 


12 


18 


5.93 


1.92 


2 


15 


67. 


0 


7.23 


2.25 


2 


15 


59.8 


6 


18 


7.21 


1.91 


2 


10 


59. 


9 


8.80 


1.98 


3 


14 


51.1 


5 


17 


9.03 


2.32 


3 


13 


46. 


9 


10.82 


2.65 


5 


14 


36.4 


3 


15 


7.36 


1.39 


2 


11 


50. 


9 


8.99 


1.70 


3 


11 


40.1 


7 


13 


4.64 


1.54 


1 


12 


64. 


3 


5.64 


1.77 


1 


12 


56.6 


8 


12 


5.60 


2.22 


1 


9 


53. 


3 


7.02 


2.20 


2 


10 


41.5 


2 


10 


5.90 


1.25 


2 


9 


41. 


0 


6.69 


1.01 


2 


8 


33.1 


1 


10 


6.11 


1.34 


1 


8 


38. 


9 


7.46 


.81 


5 


9 


25.4 


Total 


201 


93.67 




31 


153 


53. 


4 


114.45 




45 


168 


43.0 


Random 


























3 


15 


7.53 


1.55 


3 


11 


49. 


8 


'9.09 


1.50 


4 


12 


39.4 


4 


19 


10.89 


2.08 


5 


15 


42. 


7 


12.89 


1.82 


5 


15 


32.1 


11 


24 


11.92 


2.81 


3 


16 


50. 


3 


14.35 


2.08 


5 


20 


40.2 


2 


10 


5.91 


1.25 


2 


8 


40. 


9 


6.73 


1.02 


3 


8 


32.7 


7 


13 


4.92 


1.60 


1 


12 


62. 


2 


5.57 


1.72 


1 


12 


57.2 


6 


18 


7.36 


1.74 


2 


10 


59. 


1 


8.83 


2.01 


3 


15 


50.9 


1 


10 


6.04 


1.36 


1 


8 


39. 


6 


7.48 


.90 


4 


10 


25.2 


5 


17 


8.89 


2.34 


2 


12 


47. 


7 


10.87 


2.72 


3 


15 


36.1 


8 


12 


5.72 


2.12 


1 


10 


52. 


3 


7.13 


2.26 


t 


10 


40.6 


10 


23 


8.04 


4.27 


2 


21 


65. 


0 


10.51 


4.63 


2 


22 


54.3 


12 


18 


6.05 


1.81 


1 


12 


66. 


4 


7.10 


2.10 


3 


14 


60.6 


9 


22 


11.67 


3.70 


4 


17 


46. 


9 


14.02 


3.22 


6 


18 


36.3 


Total 


201 


94.94 




27 


152 


-52. 


8 


114.57 




41 


171 


43.0 



♦Computed by the formula 100-[(mean number of items in adaptive test/number 
of items in conventional test) x 100] 
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amounted to less than 1% of original test length tor the .05 termination crite- 
rion and virtually no improvement for the .01 termination criterion. 

Analysis of the percentage reduction data for individual subtests due to 
the various order-by-termination combinations revealed nearly identical results 
across ordering methods within each termination criteria. The only discernible 
trend was for the more stringent termination criterion to administer approxi- 
mately 20% more items per subtest. 

Correlation with True Achievement Levels 

Table 7 presents the values of the correlation coefficients of the Bayesian 
estimated achievement levels with the true achievement levels for the conven- 
tional and adaptive tests. Generally, these correlations were fairly consistent 
across both termination criteria and branching strategies. 

Table 7 

Correlations of Bayesian Achievement Level Estimates with the True 
Achievement Levels for 12 Subtests Using Conventional Administration, 
Adaptive Intra-Subtest Item Selection Only, and Intra-Subtest Combined 
with Three Kinds of Inter-Subtest Ordering, at Two Termination Levels 



Termination Criterion of ,05 Termination Criterion of .01 





Conven- 


Intra- 


Inter-Subtest Ordering 


Intra- 


Inter-Subtest 


Ordering 




tional 


Subtest Multiple 


No. 




Subtest Multiple 


No. 




Subtest 


Test 


Only 


R 


Items 


Random 


Only 


R 


Items 


Random 


1 


.82 


.81 


.81 


.80 


.83 


.80 


.78 


.82 


.82 


2 


.81 


.78 


.77 


.82 


.79 


.83 


.79 


.80 


.81 


3 


.83 


.77 


.79 


.83 


.80 


.79 


.79 


.79 


.79 


4 


.84 


.79 


.84 


.84 


.82 


.85 


.83 


.81 


.84 


5 


.85 


.80 


.82 


.83 


.83 


.85 


.85 


.82 


.82 


6 


.79 


.76 


.79 


.81 


.78 


.84 


.79 


.80 


.79 


7 


.73 


.66 


.64 


.73 


.71 


.70 


.64 


.67 


.67 


8 


.82 


.7? 


.79 


.82 


.80 


.84 


.82 


.78 


.79 


9 


.84 


.87 


.85 


.87 


.86 


.86 


.85 


.86 


.87 


10 


.79 


.73 


.77 


.78 


.77 


.01 


.76 


.79 


.75 


11 


.89 


.88 


.88 


.87 


.89 


.89 


.90 


.89 


• 89 


12 


.72 


.63 


.71 


.75 


.68 


.71 


.67 


.67 


.67 


Median 


.82 


.79 


.79 


.82 


.82 


.83 


.79 


.80 


.79 r 



Adaptive inf ra-subtest item selection . For the termination criterion of 
.05 with no inter-subtest branching, the correlations ranged from a low of .63 
for Subtest 12 to a high of .88 for Subtest 11. Using a termination criterion 
of .01, the correlations ranged from a low of .70 for Subtest 7 to a high of .89 
for Subtest 11. The more stringent termination criterion resulted in somewhat 
higher correlations and reduced the range of the observed correlations. 

Inter-subtest branching . When inter-subtest branching was added to the 
adaptive intra-subtest item selection, the observed correlations showed no ap- 
parent trends with respect to termination criterion or ordering procedure. 
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Although somewhat lower fidelity coefficients were noted for adaptive ver- 
sus conventional administration, it should be remembered that, on the average, 
the adaptively administered subtests used 50% fewer items to estimate the 
achievement levels of each simulee than did the conventionally administered sub- 
tests. Yet for many subtests, differences in these fidelity correlations were 
quite small; in some cases the fidelity coefficients were even slightly higher 
for the adaptive tests. 

Information 

Mean information values are shown in Table 8 for Subtest 8 [1(6)] at inter- 
vals of estimated achievement ( §) for the conventional tests, and for the adap- 
tive tests with test termination criteria of .05 and .01 using both intra-sub- 
test item selection only and inter-subtest branching, with subtest? ordered by 
highest multiple correlations (Subtest 8 was the last test administered under 
the highest multiple R ordering). These values are based on the mean informa- 

Table 8 

Mean Information Values (1(9)] and Number of Simulees (N) at Intervals 
of Estimated Achievement Levels (§) for Subtest 8 Using. Termination 
Criteria of .05 and .01 with Intra-Subtest Item Selection Only 
and Combined with Inter-Subtest Branching 



Adaptive fcntra-Subtest 

Adaptive Intra-Subtest Item Selection with 

Item Selection: Inter-Subtest Branching: 

Conventional Termination Criterion Termination Criter ion 

g Range Test .05 .01 .05 ~ .0 1 

k° Hi N I(§) N 1(g) N I(Q) ~N i(gy- 1 i(Q7 

-2.00 -1.80 - 

-1.79 -1.60 - - - - - ' - 1 .02 3 .02 

-1.59 -1.40 6 .09 6 .08 5 .08 6 .06 8 .12 

-1.39 -1.20 29 .73 19 .65 18 .54 7 -.66 11 .64 

-1.19 -1.00 14 2.06 5 2.23 7 2.21 16 2.29 11 2.07 

-•99 -.80 - - 1 3.82 4 3.92 1 3.88 4 4.10 

-.79 -.60 22 3.38 - - - - 14 3.33 11 3.28 

-.59 -.40 24 2.30 49 2.18 48 2.44 52 1.98 33 2.08 

-.39 -.20 36 1.54 49 1.48 37 1.54 30 1.46 37 1.54 

-.19 .00 41 1.76 20 1.67 38 1.72 33 1.66 39 1.77 

.01 .20 22 2.35 41 2.18 31 2.26 31 2.29 29 2.26 

.21 .40 24 3.43 25 2.98 26 3.16 29 3.24 25 3.25 

.41 .60 19 4.52 30 4.36 24 4.42 20 , 4.48 28 4.46 

.61 .80 12 " 4.97 12 4. 92 9 4.96 16 4.91 11 4.96 

.81 1.00 8 4.70 4 4.83 16 4.76 10 4.75 12 4.75 

1.01 1.20 14 4.65 10 4.74\ 18 4.75 . 9 4.69 14 4.65 



1.21 1.40 6 5.25 2 4.92\ 3 4.92 . 6 5.10 9 5.10 

1.41 1.60 15 5.31 6 5.33 \ 4 5.34 5 5.26 5 5.18 

1.61 1.80 2 . 4.55 10 4.68 7 4.68 4 4.82 2 4.32 

1.81 2.00 8 2.96 11 2.11 5 3.21 10 2.61 8 2.57 
Total 

Group 300 2.75 300 2.58 300 2.77 300 2.73 300 2.76 
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tion in the test items actually administered to each simulee, using its 6 at the 
termination of each subtest. The differences in mean information between the 
two termination conditions tended to be small, and no trends were apparent. The 
strong similarities in the data for individual subtests resulting from the two 
termination criteria suggested that little was to be gained by use of the more 
stringent .01 termination criterion. Thus, the remaining analyses were conduc- 
ted using only the .05 criterion. Appendix Tables E through P present the mean 
values of estimated information at intervals of § for the conventional and adap- 
tive subtests under all test conditions. for the termination criterion of .05. 

Adaptive intra-subtest item selection . Based on the data in Appendix Ta- 
bles E through P for the conventional test and the adaptive test with adaptive 
inter-subtest item selection only, Table 9 presents the means of the average 
subtest information values across levels of 9, their coefficients of variation, 
and their efficiencies relative to the conventional test. The mean values of 
subtest information for each subtest were virtually identical for the adaptive 
test compared to the conventional test. The coefficients of variation for the 
adaptive subtests also were quite similar to the full length conventional sub- 
tests. The efficiency of adaptive intra-subtest item selection relative to the 
full length conventional subtests appear in the last column of Table 9. These 
data suggest that there was little, if any, loss of information incurred by the 
adaptive strategy, even though the adaptive subtests averaged 50% fewer items. 

Table 9 

Mean of Average Test Information, Coefficient of Variation (CV), 
Range of Mean Test Information Across § Levels, and 
Relative Mean Efficiency to Conventional Test for Adaptive Tests with 
Intra-Subtest Item Selection Only and Termination Criterion of .05 



Adaptive Test Relative 



Conventional Test Range Mean 



Subtest 


Mean 


CV 


Mean 


CV 


Mip 


Max 


Efficiency 


1 


2.75 


.57 


2.73 


.64 


.25 


6.43 


.99 


2 


2.35 


.22 


2.46 


.35 


1.66 


6.78 


1.0'j 


3 


2.72 


.29 


2.76 


.30 


.46 


4.30 


1.01 


4 


3.25 


.14 


3.17 


.16 


.80 


5.10 


.98 


5 


3.22 


.26 


3.09 


.29 


.65 


4.31 


.96 


6 


2.17 


.18 ' 


2.08 


.24 


.24 


2.62 


.96 


7 


1.25 


.61 


1.19 


.56* 


.44 


5.03 


.95 


8 


2.75 


.53 


2.58 


.52 


.08 


5.34 


.94 


9 


4.26 


.22 


4.23 


.24 


.72 


6.86 


.99 


10 


2.46 


.51 


2.40 


.52 


.34 


12.04 


.98 


11 


5.50 


.43 


5.55 


.44 


.54 


8.61 


1.01 


12 


1.75 


.48 


1.81 


,44 


.05 


6.52 


1.03 


Total 
















Battery 


34.43 




34.05 








.99 



Figure 1 graphically depicts the overall test information curves obtained 
when the tests were administered conventionally, with only adaptive intra-sub- 
test item selection, and with inter-subtest branching in additiop to intra-sub- 
test item selection. Whereas Table 9 showed average values of information over 
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achievement levels, Figure 1 shows the mean of the average information values 
over subtests as a function of achievement level. The curves are very similar, 
with the largest differences occurring at the 8 interval -1.59 to -1.40, where 
mean information of the adaptive intra-subtest strategy was 14.66 and that of 
the conventional test was 7.82; and at the 6 intervals .61 to .80, .81 to 1.00, 
1.01 to 1.20, and 1.41 to 1.60 where the conventional test's information values 
were 37.61, 32.01, 29.10, and 26.54, respectively, and those of the adaptive 
test using only adaptive intra-subtest item selection were 30.13, 25.51, 22.10, 
and 20.04, respectively. 

Figure 1 

Mean Test Information Curves Across 12 Subtests for Conventional Test 
and for Adaptive Test Using Intra-Subtest Item Selection Only 
and Combined with Inter-Subtest Branching 
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Inter-subtest branching . Table 10 presents the mean subtest information 
values, their coefficients of variation, and efficiencies relative to the con- 
ventionally administered subtests for the adaptive tests using three methods of 
ordering subtests for inter-subtest branching. Comparison of the data in Table 
10 with that of Table 9 shows that the addition of inter-subtest branching to 
the adaptive intra-subtest item selection appeared to have a minimal effect on 
these evaluative criteria. Compare totte conventional test, the adaptive test 
with inter-subtest branching showed the same pattern of overall subtest mean 
information, as reflected in relative mean efficiencies very close to 1.00 for 
all subtests and branching conditions. , 

Figure 1 also shows the average information curve across the 12 subtests 
for the inter-subtest branching strategy using highest multiple correlations. 
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Table li 

Mean of Average Test Information, Coefficient of Variation (CV), 
Range of Mean Test Information Across 8 Levels, and Relative Mcspn 
Efficiency to Conventional Test for Adaptive Tests with Three Types 
of Inter-Subtest Branching and Termination Criterion of .05 
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Here, again, there is very little separation between the conventional test's 
curve and the curve for inter-subtest branching • One effect of intra-subtest 
item selection combined with inter-subtest branching over intra-subtest item 
election alone was w narrow the gap between the conventional test and the 
adaptive test in the 6 intervals where intra-subtest item selection alone fell 
below the conventional test. Similar results were obtained when subtests were 
ordered by number of items and randomly for use in the adaptive inter-subtest 
branching procedure. 



DISCUSSION 

This study has replicated previous findings (Brown & Weiss, 1977; Gialluca 
& Weiss, 1979) that showed that an adaptive testing strategy combining intra- 
subtest adaptive item selection with inter-subtest branching could significantly 
reduce the length of an achievement test battery while maintaining desirable 
-psychometric properties. The present study applied this test design to the same 
basic data set used by Brown and Weiss (1V77) but used a monte carlo simulation 
to assess the separate effects of adaptive intra-subtest item selection and in- 
ter-subtest branching on test length, test information, and score fidelity. In 
addition, this study investigated the effects of various subtest orderings for 
inter-subtest branching on the psychometric properties of the subtests. 

Adaptive Intra-Subtest Item Selection 

The adaptive intra-subtest item selection strategy used in this study was 
identical to that used by Brown and Weiss (1977) and Gialluca and Weiss (1979) 
in that iteas were selected for subsequent administration by selecting the item 
that provided the highest amount of psychometric information available at the 
current estimate of 6. Although such maximum information "item selection is typ- 
ically used with maximum likelihood scoring, the present application used a 
bayesian scoring method for three reasons: 

1. Maximum likelihopd scoring requires that an examinee's response vector 

fr contain at least one correct and one incorrect response before a 6 esti- 
mate can be computed; 

2. Maximum likelihood scoring does not use prior information as effectively 
as does Bayesian scoring; and 

3. Some response patterns result in a failure of the maximum likelihood pro- 
cedure to converge upon a 8 estimate. 

In general, although scores obtained from Bayesian and maximum likelihood 
methods tend to be highly correlated, they do not yield numerically identical 
results given the same data. The extent to which these two scoring procedures 
yield numerically discrepant results was reported by Kingsbury and Weiss (1979). 
Unfortunately, the issue of the most appropriate choice of scoring algorithm 
pervades implementations of IRT-based testing in general and hence should be 
addressed in future research. 

In terms of fidelity or correlations with true achievement levels, the 
adaptive intra-subtest item selection strategy using the .05 termination crite- 
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rion resulted in lower validity coefficients than tue conventionally adminis- 
tered subtests. This was likely attributable, in part, to the combined ef f ects ' 
of the Bayesian regressed achievement estimate and the smaller number of items 
used for the estimates. This, in turn, caused a restriction in the range of 
estimated achievement levels, thus lowering the correlation with true achieve- 
ment levels. The same pattern, however, did not appear when the more stringent 
.01 termination criterion was used, further supporting the hypothesized effects 
of a restricted range of Bayesian achievement level estimates. 

Test length reductions of 39% to 67% resulted from the variable termination 
criterion of the intra-subtest item selection strategy. In spite of the consid- 
erable decrease in test length, there was virtually no loss in the amount of 
psychometric information obtained from each subtest. 

Inter-Subtest Branching 

The use of prior information to determine differential subtest entry into 
subsequent subtests appeared to have little effect in terms of number of items 
administered over and above that produced by intra-subtest item selection alone. 
The extent to which this finding is generalizable across subtests with different 
characteristics and different intercorrelations remains to be investigated in 
future research. 

The addition of inter-subtest branching to the adaptive intra-subtest item 
selection generally raised the low validities obtained under the .05 termination 
condition using only adaptive intra-subtest item selection to near those ob- 
tained with conventional administration. This same effect appeared using the 
.01 termination criterion but to a lesser extent. 

A possible explanation for the small effects of prior information may lie 
in the combination of the use of linear multiple regression estimates to deter- 
mine prior ability estimates, coupled with a regressed Bayesian scoring algo- 
rithm. That is, the regression estimates themselves tend to underestimate or 
regress extreme scores and, together with the tendency of the Bayesian procedure 
to further regress the achievement estimates, might tend to offset any benefit 
arising from prior information." Perhaps the benefit of inter-subtest branching 
is not to further reduce test length but to maintain the original range of 
achievement levels against the tendency of Bayesian estimation to regress them. 
Maximum likelihood scoring might also be useful at the end of each subtest to 
obtain unregressed ability estimates for use in inter-subtest adaptive branch- 



Consistent with the results of test length reduction, mean subtest informa- 
tion was not significantly changed when inter-subtest branching was added to the 
adaptive intra-3ubtest item selection strategy. Apparently, the regression- 
based achievement estimates had the effect of selecting the same or similar 
items as those selected by the testing procedure that did not use differential 
subtest entry. This may have been caused by the rather low to moderate inter- 
correlations used in the simulation. For example, the range of intercorrela- 
tions used for the present study ranged from .14 to .53, with a mean of .34. 
Different results might be obtained with more highly correlated tests, such as 
those in the Armed Services Vocational Aptitude Battery; ASVAB-9 intercorrela- 
tions range from .12 to .75 with a mean of .50. 
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Conclusions 

The results of this study replicated and extended the findings reported by 
Brown and Weiss (1977) and Gialluca and Weiss (1979). That is, test lengths 
could be reduced by 39% to 67% using the Brown and Weiss (1977) adaptive testing 
strategy designed for test batteries. 

The design of the present study allowed the separation of the effects due 
to adaptive intra-subtest item selection from that due to inter-subtest branch- 
ing. Although the results showed that the intra-subtest item selection account- 
ed for virtually all the reduction in test length, the addition of prior infor- 
mation using inter-subtest branching, appeared to have an effect on the observed 
validities and on mean battery information. Achievement level estimates ob- 
tained using inter-subtest branching combined with adaptive intra-subtest item 
selection showed validities quite comparable to those of conventional tests 
nearly twice as long. As observed in previous studies, there was a minimal loss 
of psychometric information due to the adaptive procedure. 

Although this study replicated and extended the findings of the previous 
studies, it, too, was limited by the fact that only a small number of factors 
were included. The next step should be a large-scale simulation that systemati- 
cally varies the important sources of variation. 

Future research on inter-subtest branching should be concerned with the 
determination of the conditions under which it is maximally effective. The fac- 
tors examined should include (1) varying the factorial composition of the sub- 
tests as a way of systematically generating realistic correlation matrices; (2) 
comparing the effects of different scoring algorithms — such as Owen's Bayesian, 
modal Bayesian (which differs from Owen's in using the mode of the posterior 
distribution rather than the mean), and maximum likelihood coupled with Bayesian 
scoring (to eliminate the effects of regressed Bayesian estimates); (3) various 
subtest orderings; and (4) effects as a function of achievement level as well as 
averaged over the levels. A double cross-validation design such as used by 
Gialluca and Weiss (1979) appears to be quite useful in this regard. 

Test length in adaptive testing using a variable termination criterion is a 
function of the magnitude of the termination criterion used. The termination 
criterion that is optimal for a given situation — the magnitude of information 
that can be maintained while eliminating non-informative items — also needs to be 
addressed in future studies. The most important factor in this regard is the 
information structure of the subtests. Thus, a complete factorial design should 
also include manipulation of the subtest information curves, based on informa- 
tion curves observed in other test batteries as well as on ideal information 
curves for use in adaptive testing batteries. 
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Ttble A 

Normal Ogive Item Discrimination (a), Difficulty (b), and Pseudo-Cuesslng (c) Parameter Estimates for- the 12 Subtests 
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Subtest 6 (continued) 
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Subtest 11 (continued) 
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1.00 


2.51 


.37 


6 


1.68 


2.52 


.34 


1.21 


1.32 


.55 


7 








1.92 


1.37 


.41 


8 


1.29 


3.11 


.56 


1.64 


.03 


.54 


9 


1.58 


-.35 


.63 


.88 


1.87 


.37 


10 


2.40 


-1.01 


.90 








11 


1.23 


2.59 


.52 








12 














13 








2.52 


2.65 


.33 


14 


1.44 


.01 


.47 


11 






15 


2.41 


2.23 


.27 


1.75 


-1.36 


.61 


16 


1.67 


.53 


.57 


2.11 


1.87 


.46 


17 


2.13 


2.99 


.61 








18 


.87 


2.14 


.45 


1.06 


.17 


.47 


19 


1.19 


2.59 


.54 


.79 


1.34 


.43 


20 


2.01 


.44 


.55 


2.42 


2.71 


.57 


21 


1.45 


1.08 


.57 


1.55 


2.44 


.50 


22 


1.68 


3.17 


.59 



23 1.66 1.39 .59 



Note. 



Dashed lines Indicate that sn Item was rejected In the first phsse of the Item parameterization procedure. 
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Tabic B 

Raw Score Regression Weighte, Regression Constants, and 
Squared Standard Errors of Estimate (SSEE) for Regression 

Equations Used to Determine Differential Entry Points, 
with Subtests Ordered by Highest R . 



Subtest 



Subtest 




11 


5 


7 


9 


4 


1 6 


2 


10 12 8 


11 


562 


-Oil 
















5 


394 


178 


019 














7 


177 


174 


315 


043 












9 


114 


079 


266 


215 


-017 










4 


201 


123 


073 


055 


249 


-041 








1 


163 


201 


130 


054 


-074 


195 


-014 






6 


085 


-068 


052 


121 


300 


143 


080 -015 






2 


on 


021 


148 


178 


088 


164 


106 -008 


047 




10 


015 


035 


026 


102 


143 


052 


-045 111 


222 


-006 


12 


-051 


180 


101 


^8 


023 


-131 


108 -004 


167 


187 -Oil 


8 


044 


008 


107 


i 


013 


025 


021 072 


071 


064 121 -001 


SSEE 


755 


740 


691 


632 


668 


716 


745 764 


766 


718 772 



Note*. Eegresslon constants appear on the main diagonal* 



Table C 

Raw Score Regression Weights, Regression Constants, and 
Squared Standard Errors of Estimate (SSEE) for Regression 

Equations Used to Determine Differential Entry Points, 
with Subtests Ordered by Number of Items 



Subtest 

Subtest 11 10 9 I 12 6 5 3 7 8 2 T 



10 


220 


006 


















9 


253 


273 


-006 
















4 


225 


111 


309 


-040 














12 


241 


242 


081 


-065 


002 












6 


010 


121 


335 


186 


019 


-010 










5 


168 


043 


285 


II! 


135 


094 


022 








3 


318 


024 


049 




-016 


078 


223 -004 








7 


125 


099 


152 


031 


059 


094 


206 117 


046 






8 


012 


075 


014 


042 


136 


072 


119 050 


146 


002 




2 


-002 


166 


039 


•177 


167 


-028 


117 026 


132 


067 




1 


180 -062 


-010 


178 


103 


079 


096 156 


024 


019 


081 -016 


SSEE 


924 


729 


,697 


765 


762 


673 


538 638 


774 


712 


69# 



Note * Regression constants appear on the main diagonal* 



Table D 

Rev Score Regression Weights, Regression Constants, 
and Squared Standard Errors of Estimate (SSEE) for Regression 
Equations Used to Determine Differential Entry Points, 
with Subtests Ordered Randomly 



Subtest 



Subtest 


3 


4 


11 


2 


7 6 


1 


5 


8 


10 12 


9 


4 


425 


-040 


















U 


482 


190 -003 
















"^N 2 


143 


263 


112 


061 














7, 


219 


098 


175 


222 


041 












6 


148 


233 


-029 


036 


210 -019 












1 


170 


155 


203 


105 


032 062 


-016 










5 


226 


061 


065 


137 


224 097 


079 


006 








8 


041 


018 


035 


113 


152 087 


028 


128 


-003 






10 


018 


075 


042 


222 


109 -135 


-059 


044 


087 


-006 




12 


-054 


-128 


178 


158 


026 t008 


103 


092 


112 


179 -011 




9 


046 


J 69 


061 


036 


127 208 


-078 


195 


009 


098 017 


-006 


SSEE 


763 


726 


840 


706 


810 715 


629 


786 


771 


709 547 





Note * Regression constants appeer on the me In diagonal* 
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Table E 

Mean Informacion Values (I) at Estimated Achievement Levels (§) 
fc. Subtest 1 under All Testing Conditions 



8 Range 



Lo 



Hi 



Conven- 
tional 
Test 
~N I(g7 



Intra- 
Subtest 
Item 

Selection 



Intra-Subtest Item Selection 
with Inter-Subtest Branching 



Multiple 
R 



N 



No. of 
Items 



K r idom 
N 1(6) 



-2.C0 
-1.79 
-1.59 
-1.39 
-1.19 
-.99 
-.79 
-.59 
-.39 
-.19 
.01 
.21 
.41 
.61 
.81 
1.01 
1.21 
1.41 
1.61 
1.81 



-1.80 
-1.60 
-1.40 
-1.20 
-1.00 
-.80 
-.60 
-.40 
-.20 
.00 
.20 
.40 
.60 
.80 
1.00 
1.20 
1.40 
1.60 
1.80 
2.00 



2 

11 
12 
22 
24 
13 
11 
15 
27 
40 
8 
27 
30 
12 
46 



.23 
.58 
1.35 
2.50 
3.79 
4.94 
6.28 
5.78 
4.18 
3.26 
2.49 
2.21 
1.67 
1.41 
1.00 



10 
15 
15 
23 
14 
27 
5 
27 
26 

28 
110 



.54 
1.34 
2.66 
3.91 
5.39 
6.25 
5.58 
4.14 
3.12 

2.31 
1.22 



1 
12 

9 
17 
20 
13 
18 
13 
16 
32 
18 
42 
46 
29 

9 

5 



.12 
.58 
1.49 
2.65 
3.60 
5.48 
6.28 
5.49 
3.95 
3.12 
2.59 
2.11 
1.62 
1.20 
.87 
.75 



5 
8 
14 
15 
15 
18 
18 
11 
22 
31 
20 
28 
50 
29 
11 
3 
2 



,14 
.55 
,49 
,61 
,71 
,48 
,28 
,57 



4.22 
3.11 
2.58 
2.11 
1.59 
1.18 
.81 
.72 
.78 



3 
8 
19 
16 
24 
14 
16 
12 
25 
21 
20 
28 
52 
32 
6 
1 
2 



.13 
.41 
1.41 



.60 
.84 
.04 
.27 
.'78 
4.09 
3.08 
2.59 
2.09 
1.60 
1.20 
.83 
.72 
.80 



Total Group 300 2.75 300 2.73 300 2.71 300 2.73 300 2.71 



Table F 

Mean Information Values (I) at Estimated Achievement Levels (§) 
for Subtest 2 under All Testing Conditions 



8 Range 



Lo 



Hi 



Conven- 
tional 
Test 
N 1(9) 



Jntra- 
Subtest 
Item 
Selection 



Intra-Subtest Item Selection 
with Inter-Subtest Branching 



Multiple 
R 

T(5) 



N 



N 



No. of 
Items 



Random 



-2.00 


-1.80 


7 


3.37 






5 


3.52 


5 


3.25 


7 


2.80 


-1.79 


-1.60 


3 


4.44 


14 


5.28 


4 


6.70 


2 


'5.62 


5 


6.08 


-1.59 


-1.40 






2 


6.75 


2 


6.12 


2 


6.59 


2 


6.77 


-1.39 


-1.20 


9 


2.47 


11 


2.13 


5 


2.94 


6 


2.82 


4 


2.63 


-1.19 


-1.00 


21 


2.01 


25 


2.21 


9 


1.97 


10 


1.99 


4 


1.97 


-.99 


-.90 


8 


2.20 


7 


2.55 




2.18 


21 


2.19 


25. 


2.16 


-.79 


-.60 


14 


2.60 


26 


2.53 


19 


2.56 


23 


2.61 


18* 


2.53 


-.59 


-.40 


29 


"2.57 


43 


2.38 


32 


2.56 


18 


2.54 


27 


2.54 


-.39 


-.20 


25 


2.40 


18 


2.52 


22 


2.39 


28 


2.38 


27 


2.38 


-.19 


.00 


37 


2.48 


37 


2.79 • 


25 


2.44 


28 


2.45 


22 


2.47 


.01 


.20 


17 


2.71 


13 


2.90 


35 


• 2.74 


30 


2.75 


21 


2.78 


.21 


.40 


23 


2.89 


24 


2.84 


24 


2.89 


28 


2.89 


28 


2.89 


.41 


.60 


28 


2.69 






18 


2.76 


15 


2.69 


' 19 


2.77 


.61 


.80 


20 


2.13 






18 


2.32 


20 


2.33 


21 


2.28 


•81 


1.00 






80 


1.66 


32 


1.84 


37 


1.84 


41 


1.82 


1.01 


1.20 


59 


1.56 






17 


1.49 


14 


1.46 


23 


1.46 


1.21 


1.40 










12 


1.16 


9 


1.16 


5 


1.16 


1.41 


1.60 










3 


1.23 


4 


1.26 


1 


1.37 


1.61 


1.80 






















1.81 


2.00 






















Total 


Group 


300 


2.35 


300 


2.46* 


300 


2.44 


300 


2.40 


300 


2.41 



30 
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Table G 

Mean Information Values (I) at Estimated Achievement Levels (£) 
for Subtest 3 under All Testing Conditions 



8 Range 



Lo 



HI 



Conven- 
tional 
Test 

"h 1(e)" 



Xntra- 
Subtest 
Item 
Selection 



Intra-Subtest Item Selection 
wlth'Inter-Subtest Branching 



Multiple 
- R 

ITS) 



N 



No. of 
Items 
"n 1(1) 



Random 
N I(( 



-2.00' 


-1.80 


- 


- 


- 


- 


- 


- 


1 


.14 


_ 


_ 


-1.79 


-1.60 


4 


.78 


3 


•66 


3 


.66 


2 


.92 


5 


.70 


-1.59 


-1.40 


10 


1 .86 


8 


1.76 


5 


L m JO 


«; 
j 


1 AQ 

1 • O 7 


«; 
j 


i • j / 


-1.39 


-1.20 


3 


2.85 


12 


2.78 


10 


2.72 


9 


2.68 


15 


2,75 


-1.19 


-1.00 


17 


2.87 


' 11 


2.81 


15 


2.81 


6 


2.87 


14 


2.82 


-.99 


-.80 


23 


2.89 


13 


2.83 


17 


2.78 


28 


2.84 


16 


2.82 


-.79 


-.60 


28 


3.46 


33 


3.35 


33 


3.30 


26 


3-.31 


31 


3.35 


-.59 


-.40 


16 


4.24 


9 


4.14 


8 


4.04 


14 


4.03 


6 


4.04 


-.39 


-.20 


10 


4.24 


27 


4.16 


34 


4.21 


20 


4.17 


34 


4.21 


-.19 


.00 


25 


3.72 


21 


3.76 


14 


3.76 


20 


3.62 


27 


3.70 


.01 


.20 


23 


3.04 


41 


2.95 


36 


2.94 


35 


3.00 


26 


3.02 


.21- 


.40 


36 


2.57 


22 


"2.46 


22 


2.48 


41 


2.48 


13 


2.48 


.41 


.60 


29 


2.21 


34 


2.18 


38 


2.18 


31 


2.19 


34 


2.19 


.61 


.60 


32 


1.98 


34 


1.90 


35 


1.90 


29 


1.88 


'34 


1.38 


.81 


1.00 


15 


1.81 


* 








13 


1.70 


0 




1.01 


1.20 


13 


1.75 










9 


1.67 






1.21 


1.40 


11 


1.84 


32 


1.78' 


30 


1.76 


5 


1.82 


40 


1.78 


1.41 


1.60 


- 5 


2.13 










5 


2.04 






1.61 


1.80 














1 


2.35 






1.81 


2.00 






















Total Group 


300 


2.72 


300 


2.76 


300 


2.76 


300 


2.72 


300 


2.76 



Table H 

.Mean Information Values (I) at Estimated Achievement Levels (g) 
for Subtest 4 under All Testing Conditions 



Range 



Conven- 
tional 
Test 



Intra- 
Subtest 
Item 
Selection 



Intra-Subtest Item Selection 
with Inter-Subtest Branching 



Multiple 
R 



No. of 
Items 



Random 



Lo 


Hi 


N 


Kg) 


N 


K§) 


N 


Kg) 


N 


«§> 


N 


Ke) 


-2.00 


-1.80 


1 


.62 






1 


.44 










-1.79 


-1.60 






1 


.80 


2 


.76 


5 


.96 


1 


.86 


-1.59 


-1.40 


5 


1.70 


12 


1.62 


7 


1.83 


5 


1.70 


7 


1.80 


-1.39 


-1.20 


11 


2.51 


11 


2.52 


11 


2.50 


14 


2.52 


11 


2.58 


-1.19 


-1.00 


19 


3.19 


13 


3.12 


9 


3.14 


14 


3.08 


16 


3.13 


-.99 


-.80 


13 


3.37 


16 


3.31 


13 


3.30 


20 


3.31 


14 


3.30 


-.79 


-.60 


22 


3.30 


24 


3.22 


30 


3.20 


20 


3.20 


24 


3.20 


-.59 


-.40 


33 


3.28 


33 


3.23 


31 


3.23 


40 


3.24 


32 


3.22 


-.39 


-.20 


32 


3.41 


14 


3.42 


33 


3.42 


17 


3.'*2 


25 


3.42 


-.19 


.00 


34 


3.52 


35 


3.52 


25 


3.52 


29 


3.52 


30 


3.52 


.01 


.20 


28 


3.51 


35 


3.45 


26 


3.47 


34 


3.48 


39 


3.48 


.21 


.40 


16 


3.41 


28 


3.31 


34 


3.30 


34 


3.30 


23 


3.29 


.41 


.60 


17 


3.30 


3 


3.19 


14 


3.18 


19 


3.2G 


17 


3.18 


.61 


.80 


17 


3.20 


34 


3.10 


19 


3.09 


13 


3.11 


18 


3.09 


.81 


1.00 


17 


3.09 


1 


3.09 


10 


3.04 


13 


3.03 


11 


3.03 


1.01 


* 1.20 


8 


2.94 


17 


2.83 


2 


2*92 


13 


3.85 


10 


3.88 


1.21 


1.40 


12 


2.71 


11 


2.66 


22 


2.64 


14 


2.62 


20 


2.61 


1.41 


1.60 


5 


2.40 


4 


2.38 


4 


2.37 


3 


2.30 


1 


2.32 


1.61 


1.80 


6 


2.90 


2 


2.67 














1.81 


2'.00 


4 


5.46 


6 


5.03 


1 


7.07 


3 


6.24 


1 


5.54 


Total Group 


300 


3.25 


300 


3.17 


300 


3.15 


300 


3.16 


300 


3.17 



31 
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Table I 

Mean Information Values (I) at Estimated Achievement Levels (§) 
for Subtest 5 under All Testing Conditions 











Intra- 




Intra- 


Subtest 


Item 


Selection 






Conven- 


. Subtest 




vith Inter-Subtest 


Branching 






tional 


Item 


Multiple 


No. of 






6 Range 


Test * 


Selection 




K 


I tens 


Random 


Lo 


Hi 






n 


i\ v) 


N 


1(9) 


N 


1(9) 


N 


1(9) 


-2.00 


-1 .80 






















-1.79 


-1 .60 














*> 


• 50 


2 


.36 


-1.59 


-1 .40 


2 


AS 


c 
o 


• / u 


*% 
J 


.46 


1 A 
10 


• 71 


3 


.66 


-1.39 


-1.20 


11 


1.18 


14 


1.13 




• /l 


1 L 


i i a 


10 


l .1 j 


-1.19 


-1.00 


20 


1.75 


29 


1*79 


Q 
7 


i • J. 4. 


12 


1.79 


13 


1.79 


-.99 


-.80 


18 


2.59 


5 


2.80 


9 


1,77 


22 


2.61 


23 


2.56 


-.79 


-.60 


33 . 


3.61 • 


31 


*3.47 


* 20 


2.57 


14 


3.44 


21 


3.36 


-.59 


-.40 


18 


4.23 


13 


4.16 


18 


3.43 


21 


4.09 


26 


4.09 


-.39 


-.20 


30 


4.26 


45 


4.17 


25 


4.08 


32 


4.18 


24 


4.17 


-.19 


.00 


25 


3.85 


14 


3.90 


31 


4.17 


25 


3.88 


12 


3.78 


.01 


.20 


27 


3.41 


21 


3.40 


19 


•3.87 


27 


3.32 


30 


3.36 


.21 


.40 


25 


3.19 


38 


3.14 


33 


3.42 


19 


3.14 


20 


3.14 


.41 


.60 


27 


3.30 


11 


3.23 


29 


3.14 


25 


3.21 


29 


3.20 


.61 


.80 


16 


3.39 


23 


3.35 


20 


3.22 


26 


3.34 


30 


3.34 


.81 


1.00 


18 


3.18 


20 


3.19 


17 


3.34 


16 


3.05 


16 


3.05 


1.01 


1.20 


13 


2.74 


16 


2.58 


19 


3.06 


16 


2.61 


12 


2.58 


1.21 


1.40 


6 


2.26 


5 


2.38 


6 


2.25 


12 


2.23 


10 


2.15 


1.41 


1.60 


9 


1.98 


5 


1.96 


10 


1.91 


3 


1.84 


10 


1.86 


1.61 


1.80 










3 


1.84 


4 


1.90 


1 


1.86 


1.81 


2.00 


2 


5.80 


4 


1.96 


1 


1.98 






2 


1.97 


Total 


Group 


300 


3.22 


300 


3.09 


300 


3.08 


300 


3.05 


300 


3.03 



Table J 

Mean Information Values (I) at Estimated Achievement Levels (0) 
for Subtest 6 under All Testing Conditions 



Q Range 



Conven- 
tional 
Test 



Intra- 
Subtest 
Item 

Seiecttos 



Intra-Subtest Item Selection 
with -Inter-Subtest Branching 



Multiple 
R 



No. of 



Lo 


Hi 


N 




N^I<9) 


< N 


K8) 


N 


i(9) 


N 




-2.00 


-1.80 










1 


.04 










-1.79 


-1.60 


1 


.25 






4 


.14 


2 


.15 


3 


.14 


-1.59 


-1.40 


5 


.43 




.32 


4 


.42 


7 


.46 


4 


.31 


-1.39 


-1.20 


12 


1.23 


19 


1.03 


10 


.99 


11 


1.10 


9 


Ut2 


-1.19 


-1.00 


20 


2.07 


7 


2.18 


13 


2.05 


13 


1.9?^ 


"12 


2.08 


-.99 


-.80 


10 


2.59 


24 


2.56 


14 


2.58 


,9^ 




13 


2.56 


-.79 


-.60 


18 


2.43 


9 


2.38 


29 


2*40~~ 


^23 


2.40 


21 


2.46 


-.59 


-.40 


25 


2.23 


26 


2.18^ 




2.20 


26 


2.18 


16 


2.18 


-.39 


-.20 


32 


2.16 


1€U 


^VT2 


34 


2.10 


32 


2.10 


48 


2.10 


-.19 


.00 


23 


203-- 


- "49 


2.10 


38 


2.07 


45 


2.08 


35 


2.07 


.01 


.20 


36 


2.14 


19 


2.13 


17 


2.12 


16 


1.12 


15 


2.12 


.21 


,40 


35 


2.29 


16 


2.28 


28 


2.25 


23 


2.26 


30 


2.26 


.41 


.60 


16 


2.49 


41 


2.48 


21 


2.42 


20 


2.43 


22 


2.41 


.61 


• 80 


9 


2.66 






10 


2.60 


18 


2.60 


13 


2.60 


.81 


1.00 


14 


2.60 


18 


2.56 


21 


2.52 


17 


2*52 


26 


2.52 


1.01 


1.20 


26 


2.28 






- * 


-2.27 


15 


2.24 


11 


2.34 


1.21 


1.40 


12 


1.86 


45 


1.77 


17 


1.82 


15 


1.81 


17. 


1.83 


1.41 


1.60 


5 


1.56 






8 


1.41 


«t 5 


1.45 


4 


1.A7 


1.61 


1.80 










3 


1.17 


, 3 


1.18 


1 


1.24 


1.81 


2.00 


1 


1.86 


















Total Group 


300 


2.17 


300 


2.08 


300 


2.10 


300 


2.10 


300 


2.15 



32 
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Table K 

Mean Information Values (I) at Estimated Achievement Levels (8) 
for Subtest 7 under All Testing Conditions 



8 Range 



Lo 



Hi 



Conven- 
tional 
Test 

"5 He) 



Intra- 
Subtest 
Item 
Selecti on 



Intra-Subtest Item Selection 
with Inter-Subtest Branching 



Multiple 

R 



N 



N 



No. of 
Items 

1(6) 



Random 

"N i(eT 



-2,00 


-1.80 


- 


- 


- 


- 


- 


- 




- 


1 


^ 

.06 


-1.79 


-1.60 


17 


.31 






5 


.18 


6 


.27 


7 


.23 


-1.59 


-1.40 






16 ' 


. D 7 


j 


1 7R 




1.13 


c 
□ 


l 11 


-1.39 


-1.20 










2 


3.07 






1 


3.26 


-1.19 


-1.00 














1 


3.04 






-.99 


-.80 


6 


1.47 






3 


1.16 


10 


1.04 


6 


1.30 


-•79 


-.60 


22 


.73 


41 


.56 


23 


.58 


24 


.56 


20 


.57 


-.59 


-.40 


40 


.45 


14 


.45 


27 


.43 


34 


.42 


34 


.43 


-.39 


-.20' 


31 


.65 


89 


.66 


41 


.58 


35 


.60 


42 


.61 


-.19 


.00 


48 


1.00 






38 


1.00 


26 


.96 


31 


.96 


.01 


.20 


21 


1.47 


25 


1.42 


39 


1.47 


37 


1.38 


35 


1.38 


.21 


.40 


23 


1.77 


30 


1.87 


23 


1.79 


18 


1.82 


18 


1.81 


.41 


.60 


43 


1.92 


82 


1.90 


31 


1.89 


30 


1.89 


39 


1.89 


.61 


.80 


38 


1.88 






34 


1.83 


38 


1.83 


30 


1.83 


.81 


1.00 


6 


1.84 






26 


1.79 


23 


1.80 


20 


1.79 


1.01 


1.20 










2 


1.87 


4 


1.96 


1 


2.12 


1.21 


1.40 


















1 


2.26 


1.41 


1.60 






















1.61 


1.80 






















1.81 


2.00 


5 


4.54 


3 


3.68 


3 


7.14 


4 


6.49 


8 


9.10 


Total Group 


300 


1.25 


300 


1.19 


300 


1.31 


300 


1.28 


300 


1.41 



Table L 



Mean Information Values 
for Subtest 



lues (I) atEs£imete<r2 

s^JLuni**~*Xfi' Testing 



hlevement Levels (§) 
Conditions 



8 Range 



Conven- 
tional 
Test 



Intra- 
Subtest 
Item 
Selection 



Intra-Subtest Item Selection 
with Inter-Subtest Branching 



Multiple 
R 



No. of 
Items 



Random 



Lo 


Hi 


N 


1(6) 


N 


1(6) 


N 


i(8) 


N 




N 


i(6) 


-2.00 


-1.80 






















-1.79 


-1.60 










1 


.20 






1 


.03 


-1.59 


-1.40 


6 


.09 


6 


.08 


6 


.06 


5 


.09 


10 


.10 


-1.39 


-1.20 


29 


.73 


19 


.65 


7 


.66 


17 


.59 


13 


.48 


-1.19 


-1.00 


14 


2.06 


5 


2.23 


16 


2.29 


9 


2.60 


7 


2.78 


-.99 


-.80 






1 


3.82 


1 


3.88~ 


5 


3.97 


3 


4.06 


-.79 


-.60 


22 


3.38 






14 


3.33 


18 


3.24 


7 


3.44 


-.59 


-.40 


24 


2.30 


49 


2.18 


52 


1.98 


37 


1.96 


41 


1.93 


-.39 


-.20 


36 


1.54 


49 


1.48 




— *r4fr 


— 37 


1.46 


35 


1.46 


-.19 


.00 


_4L- 






i.67 


33 


1.66 


34 


1.68 


40 


1.66 


- .<H — 


— ;r<r 


22 


2.35 


41 


2.18 


31 


2.29 


28 


2.30 


24 


2.22 


.21 


.40 


24 


3.43 


25 


2.98 


29 


3.24 


21 


3.27 


32 


3.16 


.41 


.60 


19 


4.52 


30 


4.36 


20 


4.48 


23 


4.48 


20 


4.46 


.61 


.80 


12 


4.97 


12 


4.92 


16 


4.91 


15 


4.92 


17 


4.92 


.81 


1.00 


8 


4.70 


4 


4.83 


10 


4.75 


17 


4.77 


12 


4.75 


1.01 


1.20 


14 


4.65 


10 


4.74 


9 


4.69 


14 


4.66 


11 


4.66 


1.21 


1.40 


6 


5.25 


2 


4.92 


6 


5.10 


5 


5.06 


4 


5.14 


1.41 


1.60 


15 


5.31 


6 


5.33 


5 


5.26 


3 


5.14 


10 


5.25 


1.61 


1.80 


2 


4.55 


10 


4.68 


4 


4.82 


2 


4.84 


3 


<>.57 


1.81 


2.00 


8 


2.96 


11 


2.11 


10 


2.61 


10 


2.62 


10 


2.65 


Total Group 


300 


2.75 


300 


2.58 


300 


2.73 


300 


2.76 


300 


2.70 
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Table M 

Mean Information Values (I) at Estimated Achievement Levels ( §) 
for Subtest 9 under- All Testing Conditions 



§ Range 



Lo 



HI 



Conven- 
tional 
Test 

"N 1(8) 



Intra- 
Subtest 
Item 

Selection 



Intre-Subtest Item Selection 
with Inter-Subtest Branching 



Multiple 
R 



No. of 
Items 



Random 



N K§) 7 N 1(9) N 1(5) U I(§) 




4.84 
4.32 



Table N 

Mean Information Values (I) at Estimated Achievement Levels (§) 
for Subtest 10 under All Testing Conditions 



- 8 Range 



Lo 



HI 



Conven- 
tional 
Test 

"5 i(eT 



Intra- 
Subtest 
Item 

Selection 



Intra-Subtest Item Selection 
with Inter-Subtest Branching 



-2.00 
-1.79 
-1.59 
-1.39 

-.99 
-.79 
-.59 
-.39 
-,.19 
.01 
.21 
.41 
.61 
.81 
1.01 
1.21 
1.41 
1.61 
1.81 
Total 



-1.80 
-1.60 
-1.40 
-1.20 
- -l.Ott 
-.B0 
-.60 
-.40 
-.20 
.00 
.20 
•40 
.60 
.80 
1.00 
1.20 
1.40 
1.60 
1.80 
2.00 
Group 



7 
1 

1* 
27 
20 
22 
30 
35 
31 
31 
23 
21 
17 
7 
3 
7 

4 

300 



.30 
J. 02 
1.11 
1.09 
1.20 
1.57 
2.18 
2.74 
3.08 
2.99 
2.77 
2.66 
2.89 
3.47 
4.43 
5.59 

9.23 
2.46> 



N 



Multiple 
R 



N 



N 



No. of 
It ems 

T7e) 



Random 

"8 He) 



18 
17 
17 
29 
31 
72 
3 
32 
2d 
11 
12 
8 
5 
3 
5 
3 

300 



.47 

1.07 
1.03 
1.11 
1.44 
2.09 
2.64 
2.97 
2*89 
2.59 
2.57 
2.78 
3.64 
4.54 
5.16 
6.47 
9.58 
2.4b 



1 
1 
4 

10 
16 
26 
34 
29 
34 
36 
25 
27 
16 
8 
12 
8 
4 
3 
6 

300 



.14 
.60 
.82 
1.06 
1.03 
1.19 
1.57 
2.06 
2.65 
2.94 
2.88 
. 2.67 
2.54 
2.80 
3.44 
4,37 
5.54 
6.77 
10.52 , 
2.53 



2 

Z 

8 
35 
28 
21 
34 
38 
23 
27 
33 
12 
13 
8 
8 
3 
2 
3 

300 



.13^ 

~~730 

1.06 
1.04 
1.23 
1.50 
2.10 
2.57 
2.94 
2.88 
2.68 
2.54 
2.83 
3.57 
4.42 
5.50 
6.58 
8.17 
2.33 



4 
9 

12 
13 
29 
29 
28 
35 
23 
24 
31 
19 
13 
11 
9 
4 
4 
3 

3Q0 



.47 
.75 
1.06 
1.03 
1.18 
1.57 
2.10 
2.65 
95 
87 
63 
54 



2.79 
3.55 
4.40 
5.31 
6.27 
10.74 
2.42 



Lie 



34 
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Table 0 

Mean Information Values (I) at Estimated Achievement Levels (§) 
for Subtest 11 under All Testing Conditions 



6 Range 



Lo 



HI 



Intra* Intra-Subtest Ite* Selection 

v Convert- Subtest with Inter-Subteat Branching . 

tlonal Item Multiple No. of 

Test v Selection R Items Random 

~~W) ~S iTbT ~ ITeT n kb) jl-~*&Y 



N 



-2.00 
-1.79 
-1.59 
-1.39 

-.99 
-.79 
-.59 
-.39 
-.19 
.01 
.21 
.41 
.61 
.81 
1.01 
1.21 
1.41 
1.61 
1.81 
Total 



-1.80 
-1.60 
-1.40' 

^U00 
-.80 
-.60 
-.40 
-.20 
.00 
.20 
.40 
.60 
.80 
1.00 
1.20 
1.40 
1.60 
1.80 
2.00 
Group 



4 
8 

21 
22 
25 
21 
17 
32 
28 
22 
14 
11 
23 
20 
10 
5 
2 
2 

300 



.60 

la*--- 

L.96 

3.33 

5.U 

6.68 

7.26 

7.06 

7.20 

7.93 

8.58 

7.86 ' 

5.36 * 

3.70. 

2.73 

2.15 

2.09 

2.52 

2.92 

5.50 



2 

- T 

19 
14 
24 
24 
29 
20 
24 
30 
21 
21 
10 
11 
15 
5 
16 
2 
5 

300 



.93 
1.70 
3.36 
5.21 
6.81 
7.18 
7.03 
7.20 
8.08 
8.54 
7.45 
5.49 
3.27 
2.49 
2.13 
1.99 
2.33 
2.72 
5.55 



30 



.56 
.91 
1.76 
3.14 
5.09 
6.78 
7.18 
7.04 
7.17 
7.91 
8.53 
7.35 
5.46 
3.40 
2.58 
2.10 
2.00 
2.32 
2.77 
5.29 



16 
..12 
21 
22 
26 
31 
20 
24 
21 
11 
18 
17 
16 
15 
6 
12 
5 
7 

300 



.94 
1.72 
3.41 
5.19 
6.85 
7.18 
7.04 
7.22 
8.01 
8.53 
7.61 
5.41 
3.22 
2.53 
2.09 
1.99 
2.32 
2.72 
5.28 



3 
5 
5 
14 
12 
23 
22 
35 
24 
-26 
19 
18 
23 
17 
16 
17 
9 
7 
3 
2 

300 



.20 
.84 
.84 
1.85 
3.06 
5.16 
6.90 
7.19 
7.04 
7.19 
7.91 
8.58 
7.56 
5.40 
3.60 
2.55 
2.08 
2.00 
2.46 
2.95 
5.55 



Table ? 

Mean Information Values (I) at Estimated Achievement Levels (8) 
for Subtest 12 under All Testing Conditions 



Intra- Intra-Subtest Item Selection 

Conven- Subtest with Inter-Subtest Branching 

tlonal Item Multiple No. of 

9 Range Test Selection R Items Random 

Lo Hi N 1(5) ~ IC&7 N iTe) N 1(9) 5 1(8) 



-2.00 


, -1.80 






















-1.79 


' -1.60 






















-1.59 


-1.40 


1 


.04 










1 


.04 


1 


.03 


-1.39 


-1.20 


7 


.11 


2 


.11 


2 


.13 


3 


.15 


3 


.12 


-1.19 


-1.00 


13 


.29 


11 


.28 


11 


.27 


12 


.29 


9 


.26 


-.99 


-.80 


31 


.53 


26 


.52 


20 


.46 


23 


.51 


18 


.54 


-.79 


-.60 


33 


.86 


26 


.96 


26 


.84 


26 


.89 


32 


.86 


-.59 


-.40 


26 


1.25 


8 


1.39 


41 


1.25 


39 


1.26 


37 


1.25 


-.39 


-.20 


21 


1.77 


82 


1.58 


30 


1.69 


34 


1.66 


30 


1.69 


-.19 


.00 


34 


2.09 


11 


1.99 


32 


2.04 


30 


2.03 


25 


2.03 


.01 


.20 


36 


2.31 


46 


2.28 


35 


2.29 


31 


2.32 


29 


2.32 


.21 


.40 


21 


2.60 


13 


2.71 


19 


2.55 


22 


2.62 


19 


2.60 


.41 


.60 


23 


2.84 


23 


2.79 


31 


2.79 


19 


2.81 


28 


2.79 


• 61 


.80 


8 


2.82 


15 


2.82 


17 


2.77 


6 


2.76 


14 


2.78 


• 81 


1.00 


12 


2.63 






20 


2.51 


24 


2.55 


27 


2.53 


1.01 


1.20 


17 


2,32 


. 36 


2.37 


7 


2.20 


17 


2.29 


14 


2.30 


1.21 


1.40 


11 


2,13 






8 


2.04 


8 


2.04 


10 


2.02 


1.41 


1.60 


4 


2.16 










1 


2.13 


1 


2.10 


1.61 


1.80 


2 


2.88 










1 


2.70 


1 


2.92 


1.81 


2.00 






1 


6.52 


1 


7.86 


3 


7.69 


1 


6.30 


Total Group 


300 


1.75 


300 


1181 


300 


1.84 


300 


1.83 


300 


1.85 
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